本文提出了一种新颖的像素级分布正则化方案(DRSL),用于自我监督的语义分割域的适应性。在典型的环境中,分类损失迫使语义分割模型贪婪地学习捕获类间变化的表示形式,以确定决策(类)边界。由于域的转移,该决策边界在目标域中未对齐,从而导致嘈杂的伪标签对自我监督域的适应性产生不利影响。为了克服这一限制,以及捕获阶层间变化,我们通过类感知的多模式分布学习(MMDL)捕获了像素级内的类内变化。因此,捕获阶层内变化所需的信息与阶层间歧视所需的信息明确分开。因此,捕获的功能更具信息性,导致伪噪声低的伪标记。这种分离使我们能够使用前者的基于跨凝结的自学习,在判别空间和多模式分布空间中进行单独的对齐。稍后,我们通过明确降低映射到同一模式的目标和源像素之间的距离来提出一种新型的随机模式比对方法。距离度量标签上计算出的距离度量损失,并从多模式建模头部反向传播,充当与分割头共享的基本网络上的正常化程序。关于合成到真实域的适应设置的全面实验的结果,即GTA-V/Synthia to CityScapes,表明DRSL的表现优于许多现有方法(MIOU的最小余量为2.3%和2.5%,用于MIOU,而合成的MIOU到CityScapes)。
translated by 谷歌翻译
本文提出FogAdapt,一种用于密集有雾场景的语义细分域的新方法。虽然已经针对显着的研究来减少语义分割中的域移位,但对具有恶劣天气条件的场景的适应仍然是一个开放的问题。由于天气状况,如雾,烟雾和雾度,加剧了域移位的场景的可见性,从而使得在这种情况下进行了无监督的适应性。我们提出了一种自熵和多尺度信息增强的自我监督域适应方法(FOGADAPT),以最大限度地减少有雾场景分割的域移位。由经验证据支持,雾密度的增加导致分割概率的高自熵性,我们引入了基于自熵的损耗功能来引导适应方法。此外,在不同的图像尺度上获得的推论由不确定性组合并加权,以生成目标域的尺度不变伪标签。这些规模不变的伪标签对可见性和比例变化具有鲁棒性。我们在真正的雾景场景中评估了真正的清晰天气场景模型,适应和综合非雾图像到真正的雾场景适应情景。我们的实验表明,FogAdapt在有雾图像的语义分割中的目前最先进的情况下显着优异。具体而言,通过考虑标准设置与最先进的(SOTA)方法相比,FogaDATK在Foggy苏黎世上获得3.8%,有雾的驾驶密集为6.0%,而在Miou的雾化驾驶的3.6%,在Miou,在MiOOP中改编为有雾的苏黎世。
translated by 谷歌翻译
Several self-supervised representation learning methods have been proposed for reinforcement learning (RL) with rich observations. For real-world applications of RL, recovering underlying latent states is crucial, particularly when sensory inputs contain irrelevant and exogenous information. In this work, we study how information bottlenecks can be used to construct latent states efficiently in the presence of task-irrelevant information. We propose architectures that utilize variational and discrete information bottlenecks, coined as RepDIB, to learn structured factorized representations. Exploiting the expressiveness bought by factorized representations, we introduce a simple, yet effective, bottleneck that can be integrated with any existing self-supervised objective for RL. We demonstrate this across several online and offline RL benchmarks, along with a real robot arm task, where we find that compressed representations with RepDIB can lead to strong performance improvements, as the learned bottlenecks help predict only the relevant state while ignoring irrelevant information.
translated by 谷歌翻译
Quantitative cephalometric analysis is the most widely used clinical and research tool in modern orthodontics. Accurate localization of cephalometric landmarks enables the quantification and classification of anatomical abnormalities, however, the traditional manual way of marking these landmarks is a very tedious job. Endeavours have constantly been made to develop automated cephalometric landmark detection systems but they are inadequate for orthodontic applications. The fundamental reason for this is that the amount of publicly available datasets as well as the images provided for training in these datasets are insufficient for an AI model to perform well. To facilitate the development of robust AI solutions for morphometric analysis, we organise the CEPHA29 Automatic Cephalometric Landmark Detection Challenge in conjunction with IEEE International Symposium on Biomedical Imaging (ISBI 2023). In this context, we provide the largest known publicly available dataset, consisting of 1000 cephalometric X-ray images. We hope that our challenge will not only derive forward research and innovation in automatic cephalometric landmark identification but will also signal the beginning of a new era in the discipline.
translated by 谷歌翻译
We propose RANA, a relightable and articulated neural avatar for the photorealistic synthesis of humans under arbitrary viewpoints, body poses, and lighting. We only require a short video clip of the person to create the avatar and assume no knowledge about the lighting environment. We present a novel framework to model humans while disentangling their geometry, texture, and also lighting environment from monocular RGB videos. To simplify this otherwise ill-posed task we first estimate the coarse geometry and texture of the person via SMPL+D model fitting and then learn an articulated neural representation for photorealistic image generation. RANA first generates the normal and albedo maps of the person in any given target body pose and then uses spherical harmonics lighting to generate the shaded image in the target lighting environment. We also propose to pretrain RANA using synthetic images and demonstrate that it leads to better disentanglement between geometry and texture while also improving robustness to novel body poses. Finally, we also present a new photorealistic synthetic dataset, Relighting Humans, to quantitatively evaluate the performance of the proposed approach.
translated by 谷歌翻译
Denoising diffusion models hold great promise for generating diverse and realistic human motions. However, existing motion diffusion models largely disregard the laws of physics in the diffusion process and often generate physically-implausible motions with pronounced artifacts such as floating, foot sliding, and ground penetration. This seriously impacts the quality of generated motions and limits their real-world application. To address this issue, we present a novel physics-guided motion diffusion model (PhysDiff), which incorporates physical constraints into the diffusion process. Specifically, we propose a physics-based motion projection module that uses motion imitation in a physics simulator to project the denoised motion of a diffusion step to a physically-plausible motion. The projected motion is further used in the next diffusion step to guide the denoising diffusion process. Intuitively, the use of physics in our model iteratively pulls the motion toward a physically-plausible space. Experiments on large-scale human motion datasets show that our approach achieves state-of-the-art motion quality and improves physical plausibility drastically (>78% for all datasets).
translated by 谷歌翻译
Data scarcity is a notable problem, especially in the medical domain, due to patient data laws. Therefore, efficient Pre-Training techniques could help in combating this problem. In this paper, we demonstrate that a model trained on the time direction of functional neuro-imaging data could help in any downstream task, for example, classifying diseases from healthy controls in fMRI data. We train a Deep Neural Network on Independent components derived from fMRI data using the Independent component analysis (ICA) technique. It learns time direction in the ICA-based data. This pre-trained model is further trained to classify brain disorders in different datasets. Through various experiments, we have shown that learning time direction helps a model learn some causal relation in fMRI data that helps in faster convergence, and consequently, the model generalizes well in downstream classification tasks even with fewer data records.
translated by 谷歌翻译
图像二进制技术通常用于增强嘈杂和/或退化的图像来迎合不同文档图像Anlaysis(DIA)应用(如单词斑点,文档检索和OCR)。大多数现有技术都集中在将像素图像馈送到卷积神经网络中以完成文档二进制化,这在使用不完全减压的情况下需要处理的压缩图像时可能不会产生有效的结果。因此,在本研究论文中,通过使用双重鉴别器生成对抗网络(DD-GAN),提出了使用JPEG压缩图像的文档图像二进制的想法。在这里,两个歧视者网络 - 全球和本地工作在不同的图像比率上,并将焦点损失用作发电机损失。提出的模型已通过不同版本的DIBCO数据集进行了彻底的测试,该数据集具有诸如孔,擦除或弄脏的墨水,灰尘和放错地方的挑战。在时间和空间复杂性方面,该模型被证明是高度鲁棒,有效的,并且还导致了JPEG压缩域中的最新性能。
translated by 谷歌翻译
全身追踪器用于监视和安全目的,例如人跟踪机器人。在中东,统一的人群环境是挑战最新跟踪器的常态。尽管过去文献中记录的跟踪器技术有了很大的改进,但这些跟踪器尚未使用捕获这些环境的数据集进行了培训。在这项工作中,我们在统一的人群环境中开发了一个带有一个特定目标的注释数据集。该数据集是在四种不同的情况下生成的,在四种不同的情况下,目标主要是与人群一起移动,有时会与它们阻塞,而其他时候,相机的目标视图在短时间内被人群阻止。注释后,它用于评估和微调最新的跟踪器。我们的结果表明,与初始预训练的跟踪器相比,基于两个定量评估指标的微调跟踪器在评估数据集上的性能更好。
translated by 谷歌翻译
端到端(E2E)模型已成为最新语音识别系统的默认选择。此类型号经过大量标记数据的培训,这些数据通常无法用于低资源语言。诸如自我监督学习和转移学习的诺言之类的技术尚未在培训准确的模型中有效。另一方面,在各种域和扬声器集合上收集标记的数据集非常昂贵。在这项工作中,我们通过公共资料中的印度语言,特别是来自印度广播电台的公共档案馆的印度语言的``采矿''文本和音频对展示了这些方法的廉价和有效替代方案。作为关键组件,我们将Needleman-Wunsch算法调整为与相应的音频片段对齐句子,并给定长音频和其转录本的PDF,同时由于OCR,无关紧要的文本和未转录的语音而对错误进行了强大的态度。因此,我们创建了Shrutilipi,这是一个数据集,其中包含超过6,400个小时的12个印度语言标签的音频,总计为495万个句子。平均而言,Shrutilipi导致2.3倍增加了公开可用的标签数据。我们在12种语言中与21种人类评估者建立了Shrutilipi的质量。我们还根据代表区域,说话者和提到的实体建立了Shrutilipi的多样性。值得注意的是,我们表明,将Shrutilipi添加到WAV2VEC模型的训练集中,导致在Indicsuperb基准上的7种语言中,平均降低了5.8 \%。对于具有最多基准的印地语(7),平均水平从18.8%下降到13.5%。这种改进扩展到有效的模型:对于构象异构体模型(比WAV2VEC小10倍),我们显示出2.3%的下降。最后,我们通过证明对其进行训练的模型对嘈杂的输入更强大,证明了Shrutilipi的多样性。
translated by 谷歌翻译